Measuring physical objects and presenting virtual articles

ABSTRACT

An image of an environment including an object (i.e., object in reality, physical object) can be captured and be displayed by a computing device. The size of an image representation of the object can be determined. Information about a physical size of the object and about a distance between the computing device and the object in the environment can be obtained/determined. A comparison of the size of the image representation of the object and the physical size of the object can be used to calculate a scale for converting between an actual size of a real-world item and a size of the item as displayed. The information about the distance can optionally be used to determine/adjust/modify the scale. A virtual article depicting/representing a real-world article/item/object (i.e., real-world correspondence) can be displayed with the captured image. The virtual article displayed can be scaled to an appropriate size using the scale.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/613,469, filed Sep. 13, 2012, entitled “MEASURING PHYSICAL OBJECTSAND PRESENTING VIRTUAL ARTICLES,” which is hereby expressly incorporatedherein by reference in its entirety.

BACKGROUND

Computing devices are often used to make purchases. A user of acomputing device can use his/her computing device to make an onlinepurchase. There are various electronic marketplaces for the user toengage in online shopping. Online electronic marketplaces often providepictures, descriptions, and other information for their products. Unlikephysically shopping at a physical retail store, online shopping does notallow the user to examine the item in real life. For some items, such assoftware, providing just a picture and description is sufficient becausethe user does not need to examine the item in person. However, for someitems, such as furniture, providing pictures and descriptions might givethe user insufficient, inaccurate, and/or incomplete information. Forexample, a picture of a couch can appear smaller in the picture than itactually is in reality. Moreover, sometimes an item that the userbelieves to be appropriate and desirable based on online pictures anddescriptions can turn out to be inappropriate or undesirable. The itemcan be inappropriate, for example, in that it does not fit in or matchwith the environment and surroundings where it is to be placed. The itemcan be undesirable, for example, if it does not look as good asportrayed in the online pictures and/or if it does not meet the user'sexpectations. As such, the user might need to return or exchange theinappropriate or undesirable item at the online electronic marketplace.This can be a waste of time, effort, and resources for both the user andthe online electronic marketplace. Accordingly, it would be beneficialto provide the user with additional information, perspective, and/orinteractivity in viewing, examining, and/or considering one or moreitems of interest.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 illustrates an example environment in which aspects of thevarious embodiments can be utilized;

FIG. 2 illustrates an example embodiment for measuring objects inreality and viewing virtual articles;

FIG. 3 illustrates an example embodiment for measuring objects inreality and viewing virtual articles;

FIG. 4A illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles;

FIG. 4B illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles;

FIG. 4C illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles;

FIG. 5 illustrates an example method embodiment for measuring objects inreality and viewing virtual articles;

FIG. 6 illustrates an example method embodiment for measuring objects inreality and viewing virtual articles;

FIG. 7 illustrates an example device that can be used to implementaspects of the various embodiments;

FIG. 8 illustrates example components of a client device such as thatillustrated in FIG. 7; and

FIG. 9 illustrates an environment in which various embodiments can beimplemented.

DETAILED DESCRIPTION

Systems, devices, and methods in accordance with various embodiments ofthe present disclosure overcome one or more of the above-referenced andother deficiencies in conventional approaches to providing users withinformation about physical objects using electronic data. In particular,various embodiments can provide one or more virtual articles for displayusing an electronic device, where each virtual article depicts one ormore corresponding physical objects, herein referred to as “real-worldcorrespondences.” For example, a virtual article can be a graphicaldepiction and/or representation of a physical object, such as a piece offurniture, a decoration, a piece of art (e.g., sculpture, sketch,painting, photograph, poster, etc.), an appliance, a textile (e.g., rug,curtains, bed sheet, etc.), a fixture (e.g., a ceiling fan, a light, awindow, a mirror, a fireplace, etc.), a wall, a household item, etc.

In some embodiments, a virtual article can be scaled in size on thedisplay screen with respect to other content displayed on the screen.For example, the scaling in size of the virtual article can be based atleast in part upon measuring the size(s) of one or more objects inreality (i.e., physical objects) also being presented on the displayscreen. An object in reality (i.e., physical object in an environment)can be any real-world item, such as a piece of furniture, a decoration,a piece of art (e.g., sculpture, sketch, painting, photograph, poster,etc.), an appliance, a textile (e.g., rug, curtains, bed sheet, etc.), afixture (e.g., a ceiling fan, a light, a window, a mirror, a fireplace,etc.), a wall, a household item, etc. Continuing with the scaling of thevirtual article, in some embodiments the scaling can be based at leastin part upon comparing a physical size of an object in reality (e.g., inan environment) with a size of an image representation of the object asdisplayed on the screen. For example, if an object in reality is 100units long but (its image representation) appears to be 10 pixels longon the display, then the system can determine that 1 pixel of the imagerepresentation on the display screen corresponds to 10 units in reality.The system can thus scale the virtual article on screen to be a sizethat is one-tenth (i.e., 1/10) in pixels of what the correspondence inreality would be in actual units. As such, a scale (e.g., measurementscale), useful for converting between a size of an item in reality and asize of the item on the display screen, can be based at least in partupon comparing the actual size of an object in reality with the size ofan image representation of the object as it would appear on the displayscreen. The measurement scale can be used to scale the virtual articlein size to depict its real-world correspondence as it would appear inreality.

In some embodiments, the size of an object on the display screen canchange as the distance between the object in reality and the computingdevice changes. For example, as the distance between the object inreality and the computing device decreases, the size of the object aspresented on the display screen will increase; at this decreaseddistance, the size of the virtual article presented on the displayscreen should be bigger as well. Likewise, if the distance between theobject in reality and the device increases, then the size of the objecton the display will decrease; at this increased distance, the size ofthe virtual article presented on the display screen should be smaller.As such, the measurement scale can depend at least in part upon thedistance between the device and the object in reality.

In some embodiments, the various embodiments of the present disclosurecan improve a shopping experience for a user. For example, a user canuse a camera on his/her computing device to capture an image (e.g.,picture, video) of his/her living room (i.e., environment), which caninclude a back wall, a coffee table on the left side of the back wall,and a right wall attached and perpendicular to the back wall. The imageof the living room can be displayed on a display screen on the computingdevice. The user can desire to buy a chair to place against the backwall, between the coffee table on the left and the wall on the right.The user's computing device can determine that the length of the coffeetable is 200 pixels on the screen. The device can obtain information(e.g., from a distance measuring device, from the user) that the devicewas 5 feet away from the coffee table and/or back wall when the imagewas captured. The device can also obtain information that the real,physical length of the coffee table is 4 feet (e.g., determine thephysical length based at least in part upon the information obtainedabout the distance between the device and coffee table, determine thephysical length by utilizing object recognition technology, and/orobtain the physical length from the user, etc.). The device candetermine a measurement scale specifying that at a distance of 5 feet, 1foot in reality is equal to 50 pixels (and vice versa). The device canfurther determine (e.g., via image processing) the space between thecoffee table and the right wall to be 250 pixels. Based on themeasurement scale, the device can determine that only real-worldcorrespondences that are 5 feet wide or less can fit in the spacebetween the coffee table and the right wall. An electronic marketplacecan utilize this information and provide to the user a list of chairs(i.e., real-world correspondences) that are at most 5 feet in width. Thedevice can present on the screen a virtual article for each respectivechair on the list, each virtual article being displayed in the space inthe living room and being scaled appropriately based on the measurementscale (at most 250 pixels wide). As such, the user can see a virtualrepresentation on the screen of how the chair would look in the spacebetween the coffee table and right wall in his/her living room. The usercan thus be in a better position to make a purchase.

FIG. 1 illustrates an example environment in which aspects of thevarious embodiments can be utilized. The example environment 100 cancomprise a computing device 102, at least one camera 104 (i.e., imagecapture component) on the computing device 102, a display screen 110 onthe computing device 102, and an object in reality (i.e., physicalobject) 106. The computing device 102 can comprise the at least onecamera 104 and the display screen 110. A user 108 of the computingdevice 102 can use the camera 104 to capture an image (e.g., picture,video, real-time video feed, etc.) of a perspective viewable to thecamera 104. The perspective (i.e., view) in the captured image caninclude the object in reality 106.

The object in reality 106 can be any real-world item viewable by the atleast one camera 104. For example, the object in reality (i.e., physicalobject) 106 can be a plant pot, as shown in FIG. 1. In some embodiments,the object in reality 106 can be a piece of furniture, a decoration, apiece of art (e.g., sculpture, sketch, painting, photograph, poster,etc.), an appliance, a household item, etc. The object in reality 106 inthe captured image can be presented on the display screen 110 of thecomputing device 102. In some embodiments, the at least one camera 104can capture an image of a view which includes multiple objects inreality.

As illustrated in the example environment 100, there can be a distance112 between the computing device 102 and the object in reality 106. Thecomputing device 102 can use various technologies available to thedevice 102 to obtain information about the distance 112. In someembodiments, the computing device 102 can have an ultrasonic sensor, aninfrared (IR) sensor (e.g., IR camera), etc. For example, the at leastone camera 104 on the computing device 102 can include an IR camera suchthat the device 102 can utilize the IR camera to measure the distance112 between the device 102 and the object 106. In some embodiments, thecomputing device 102 can obtain information about the distance 112 fromthe user 108. For example, the device 102 can prompt the user 108 toinput the distance 112 to the object 106. In another example, the device102 can instruct the user 108 to position himself/herself a specifieddistance 112 (e.g., based on a specified/default distance parameter)away from the object 106, such that the specified distance can bereadily obtained by, or be already “known” to, the device 102 (e.g., viathe distance parameter, which can be stored on or accessible to thedevice).

FIG. 2 illustrates an example embodiment for measuring objects inreality and viewing virtual articles. The example embodiment can includea computing device 202 that comprises at least one camera 204 and adisplay screen 210. The computing device 202 can use the at least onecamera 204 to capture an image of a view behind the computing device202. There can also be one or more objects in reality, such as a plantpot 206 and/or an end table 208, in the view captured by the at leastone camera 204. The captured image of the view, including the object(s)in reality (e.g., 206 and 208), can be presented on the display screen210 of the computing device 202.

As shown in FIG. 2, there can be a distance 212 between the computingdevice 202 and the one or more objects in reality (e.g., 206 and 208).The distance 212 can be a linear distance in a substantially horizontalplane parallel, within an allowable deviation, to the floor/ground. Thedistance can also be, for example, substantially orthogonal (i.e.,orthogonal within an allowable deviation) to the back wall 216. In someembodiments, the distances from the device 202 to each of multipleobjects in reality can be similar (e.g., within an allowable deviationfrom one another); these distances can also be averaged to result in thedistance 212. For example, if the device 202 is moved left in a verticalplane parallel to the back wall 216, then when the device 202 is infront of the plant pot 206, the distance between the device 202 and thepot 206 (i.e., “pot distance”) will be similar to distance 212.Likewise, if the device 202 is moved right in the vertical planeparallel to the back wall 216, then when the device 202 is in front ofthe end table 208, the distance between the device 202 and the end table208 (i.e., “table distance”) will be similar to distance 212 (and to thepot distance). In some embodiments, when there are multiple objects, thedistance 212 can be determined by averaging or taking the median of thedistances between the device and each of the multiple objects in reality(e.g., pot distance and table distance). As mentioned above, informationabout distance (e.g., the measurement for distance 212) can be obtainedin various ways, such as by utilizing an ultrasonic sensor, an IRsensor, a specified/default distance parameter, information from theuser, or any combination thereof.

In some embodiments, an object in reality can be a wall, such as theback wall 216. The wall can include one or more edges, such as where thewall meets an adjacent wall. In some embodiments, the computing device202 can utilize image processing to determine/obtain information aboutthe wall (e.g., the shape of the wall, one or more dimensions of thewall, one or more boundaries of the wall, etc.). For example, thecomputing device 202 can capture an image of the back wall 216 and useedge/line detection applied to the captured image of the back wall 216to determine one or more boundaries of the back wall 216, from which theshape, width, etc. of the back wall 216 can be inferred. Furthermore,the distance 212 between the device 202 and an object in reality can bethe distance between the device 202 and the back wall 216.

The one or more objects in reality can be used for determining ameasurement scale (which, in some embodiments, can comprise a conversionalgorithm and/or ratio) useful for converting (back and forth) betweensizes of objects in reality and their respective sizes on the displayscreen. In some embodiments, an object in reality can be providedintentionally for such purposes. For example, in some embodiments, arectangular piece of paper (e.g., cardboard, plastic, etc.) of a knownsize (e.g., width and/or height) can be provided for use with thevarious embodiments of the present disclosure. The size of the paper inthis example can be known to the device 202, or known to the user suchthat the device 202 can obtain the size from the user. Continuing withthe example, the paper can also be of a specific color and/or pattern tofacilitate detection by the device 202 (e.g., by the camera 204 and/orby image processing on the device 202). The paper (i.e., object inreality) can be placed on a wall (e.g., back wall 216) at a certaindistance (e.g., distance 212) away from the device 202. The at least onecamera 204 can capture an image of the paper on the back wall 216 andpresent the paper on the display screen 210. Since the actual size ofthe paper in reality is known (or can be obtained), the device 202 cancompare the actual size of the paper with the size of the paper asdisplayed on the display screen 210 to determine a measurement scale(e.g., based at least in part upon a conversion algorithm and/or ratioto be applied) for converting (back and forth) between the size of anobject in reality and the size of the object as it would appear on thedisplay screen 210. Moreover, since the distance between the paper andthe device 202 can be determined/obtained (as discussed above), themeasurement scale (e.g., the conversion algorithm/ratio) can be adjustedin response to any changes to the distance (e.g., if the device 202moves closer or farther away from the paper).

In some embodiments, any object in reality, such as the plant pot 206,the end table 208, the back wall 216, or any combination thereof can beused for determining the measurement scale. For example, an object suchas the back wall 216 can be used for determining the measurement scale.In some embodiments, the size (e.g., width) of the back wall 216 can bedetermined (obtained) from the user to be 11 feet. In some embodiments,the size (e.g., width) back wall 216 can be determined to be 11 feetbased at least in part upon utilizing the distance between the wall 216and the device 202. Continuing with the example, the device 202 candetermine that the size (width) of the back wall 216 as presented on thedisplay screen 210 is 1100 pixels. Accordingly, a measurement scale(e.g., algorithm/ratio) can be determined, specifying that, assuming thedistance 212 is constant, each feet in reality corresponds to 100 pixelson the device 202 and vice versa. It should follow that if the plant pot206 is one foot tall by one foot wide, then it will be 100 pixels by 100pixels on the display screen 210. Likewise, if the end table 208 is 2.5feet wide and 1.5 feet tall in reality, then it will be 250 pixels wideand 150 pixels tall on the display screen 100. Moreover, informationabout the distance 212 can be determined/obtained such that themeasurement scale can be adjusted appropriated if there are any changesto the distance 212. Based on the measurement scale, any size of anobject in reality can be converted to its respective size on the displayscreen 210 and any size of a virtual article on the display screen 210can be converted to its respective size in reality.

In some embodiments, there can be a separation of space between two ormore objects in reality. For example, as shown in FIG. 2, there can be adistance 214 between the plant pot 206 and the end table 208. Using thedetermined measurement scale, the distance 214 in reality can becalculated from the width of the empty space displayed on the displayscreen 210. For example, image processing (e.g., line/edge detection,object recognition, etc.) running on the device 202 can be used detectthe plant pot 206 and the end table 208 displayed on the display screen210. Then the measurement scale can be applied on the number of pixelsin between the pot 206 and the table 208 on the screen 210 to obtain thesize/width of the space (i.e., distance 214) in reality. For example, ifthe number of pixels between the pot 206 and table 208 is determined tobe 650 pixels, then distance 214 will be 6.5 feet in reality. In someembodiments, information about the distance 214 can beprovided/supplemented by the user to the device 202.

FIG. 3 illustrates an example embodiment for measuring objects inreality and viewing virtual articles. In the example embodiment, therecan be a computing device 302 comprising at least one camera 304 and adisplay screen 310. Having determined the size of the free/empty spacebetween two or more objects (e.g., pot 206 and table 308) presented inthe display screen 310 and/or the size of the empty space in reality,the example embodiment can provide one or more virtual articles 312 tobe presented on the display screen 310 in conjunction with the imagecaptured by the camera 304 of the view behind the device 302 includingthe objects in reality (e.g., 306 and 308).

The example embodiment can elect to present the virtual article 312based at least in part upon the virtual article's size on screen (e.g.,pixel size) and/or its corresponding size in reality (e.g., in feet,meters, etc.). For example, the example embodiment can choose thevirtual article 312 because it had a pixel width of 625 pixels, which isless than the pixel width between the pot 306 and the table 308 (e.g.,650 pixels) displayed on the screen 310. Additionally or alternatively,the example embodiment can obtain information about the virtual article312, which can include a physical dimension of a real-worldcorrespondence of the virtual article 312. For example, if the virtualarticle 312 represented a product/object in reality 314, the exampleembodiment can obtain a product detail of the object in reality 314corresponding to and/or represented by the virtual article 312; theproduct detail can include an actual, physical width of the object inreality 314. If the physical width of the object in reality 314 is 6.25feet in this example, then the object 314 can fit between the pot 306and table 308 because the distance between the pot 306 and the table 308is 6.5 feet.

By providing the virtual article 312 to be presented on the displayscreen 310 in conjunction with the captured image of the view andobjects in reality (e.g., 306 and 308), a user of the device 302 canmore easily imagine how a real-world correspondence 314 of the virtualarticle 312 would appear, without having to physically obtain thecorrespondence 314. As illustrated in the example embodiment of FIG. 3,the virtual article 312 can correspond to a couch in reality 314.However, the user does not need to actually get/buy the couch 314,assemble it, and physically place it in his/her living room to get asense of how the couch 314 would look and/or to determine whether he/shelikes the couch 314. If the couch 314 does not fit, match, and/or if theuser does not like the couch 314, then he/she might still have to returnor exchange the couch 314, causing much trouble and effort. Instead, inaccordance with the present disclosure, the user can just enable his/hercomputing device 302 to provide a virtual article 312 corresponding tothe couch 314 to be presented along with his/her living room and otheritems (e.g., 306 and 308) on the display screen 210 of the device 302.As such, the time, effort, and/or resources of the user and the partyproviding (e.g., selling) the couch 314 are saved.

FIG. 4A illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles. The example device embodiment400 can comprise a display screen 410. In FIG. 4A, the example deviceembodiment 400 can present one or more virtual articles (e.g., 412, 414,416, 418, 428, etc.) that fit within the environment (e.g., that fitwithin the space between the plant pot and the end table). For example,there can be a shopping catalog that contains various products. If theuser desires to view couches, the shopping catalog can be filtered toinclude only those couches that fit within the environment. The user canselect a couch 412 to view in conjunction with the environment (e.g.,overlaying his/her room). Other couches (e.g., 414, 416, 418, 420, etc.)that fit can be provided to the user as well. There can also beadditional information 422 provided to the user for the currentlyselected item 412. The information 422 can include a productdescription, detailed information about the product, dimension (e.g.,width, length, height, weight) of the product, product location,availability, price, color, pattern, style, etc. There can also bebuttons for user interaction, such as a button 424 to purchase the item412.

A person of ordinary skill in the art would understand that variousother elements and interactivity can be provided to the user, such asadditional buttons, addition information, and addition functions. Forexample, the device embodiment can allow the user to change theorientation of the item 412 presented with the environment. In anotherexample, the user can move the item 412 and/or other objects within theenvironment. In a further example, the user can zoom in and out. In someembodiments, the device embodiment can allow the user to change thelighting of the environment presented on the display screen 410.

FIG. 4B illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles. The example device embodiment430 can take into consideration the any color and/or pattern schemeswithin the environment. For example, the user can instruct the device totake into account any color/pattern scheme(s) present within theenvironment. In some embodiments, image processing can be performed withrespect to the captured image of the environment and the one or moreobjects in reality, in order to detect/determine the color/patternscheme(s) present. If the device embodiment determines that a particularcolor/pattern scheme(s) is present, then the device embodiment canattempt to provide articles that match the color/pattern scheme(s).

For example, as shown in FIG. 4B, the plant pot and the end table have aparticular color/pattern scheme. The device can detect and determine thecolor/pattern scheme, and filter the shopping catalog such that onlythose articles (e.g., 442, 444, 446, 448, 450, etc.) matching the schemewill be provided to the user.

FIG. 4C illustrates an example device embodiment for measuring objectsin reality and viewing virtual articles. The example device embodiment460 can take into account one or more styles of the environment and theobjects in reality. For example, the user can instruct the device totake into account any style scheme(s) present within the environment. Insome embodiments, image processing (e.g., line/edge detection, objectrecognition) can be utilized to detect/determine the style(s) associatedwith the environment and/or the objects. If the device embodimentdetermines that a particular style scheme(s) is present, then the deviceembodiment can attempt to provide articles that match the stylescheme(s).

For example, as shown in FIG. 4B, the plant pot and the end table have aparticular style scheme (e.g., a straight edge style, an angular style,etc.). The device can detect and determine the style scheme, and filterthe shopping catalog such that only those articles (e.g., 472, 474, 476,478, 480, etc.) matching the scheme will be provided to the user. Inaddition, in some embodiments, one or more schemes related to style,color, pattern, or any combination thereof can be taken intoconsideration.

FIG. 5 illustrates an example method embodiment 500 embodiment formeasuring objects in reality and viewing virtual articles. It should beunderstood that there can be additional, fewer, or alternative stepsperformed in similar or alternative orders, or in parallel, within thescope of the various embodiments unless otherwise stated. The examplemethod embodiment 500 can start with capturing an image of anenvironment including an object, at step 502. The image can be capturedusing at least one camera on a computing device. At step 504, theexample method 500 can display the captured image on a display of thecomputing device. For example, the captured image can be at least onevideo image frame of a video feed including the environment and theobject, the video feed can be displayed on the display in real-time. Inanother example, the captured image can be a still picture of theenvironment and the object. In some embodiments, the display cancomprise a touch sensor; in other words, the display can be a touchscreen.

At step 506, the example method embodiment 500 can determine a size ofan image representation of the object based on the captured image. Insome embodiments, the size of the image representation of the object canbe determined (e.g., estimated) based on the captured image, withouthaving to actually present (e.g., render) the object on the display. Forexample, the method can determine from the captured image that if theobject was to be presented on display, it would appear 100 pixels wideand/or 200 pixels high. In some embodiments, the example method candetermine a size of the image representation of the object by presentingthe object on the display. For example, the object presented on thedisplay can be determined to be 100 pixels wide and/or 200 pixels high.In some embodiments, image processing (e.g., line/edge detection, objectrecognition) can be utilized in determining the size of the imagerepresentation of the object.

At step 508, the example method 500 can determine a distance between thecomputing device and the object. In some embodiments, the computingdevice can utilize an ultrasonic sensor, an infrared (IR) sensor, thecamera, and/or other distance measuring utilities (e.g., stereoscopiccameras, RGB cameras for sensing UV light, laser beams, sonar pulses,etc.) to determine the distance between the device and the object. Insome embodiments, the distance between the computing device and theobject can be determined/obtained from the user and/or from aspecified/default distance parameter (e.g., which can be stored in amemory of the device). In some embodiments, the distance between thecomputing device and the object can be a linear distance having an axissubstantially orthogonal (i.e., within an allowable deviation from beingorthogonal) with respect to a vertical plane such as a back wall of theenvironment.

The example method 500 can calculate a physical size of the object basedon the determined distance and the determined size of the imagerepresentation of the object, at step 510. The example method 500 cancalculate a scale for the captured image based on the calculatedphysical size of the object and the determined size of the imagerepresentation of the object, at step 512. For example, the scale can beuseful for converting back and forth between a pixel size on the displayand a feet (or meter, etc.) measurement in reality (e.g., in theenvironment). In some embodiments, the calculating of the scale can bebased at least in part upon comparing a physical size of the object inreality with the determined size of image representation of the objectas presented on the display.

In some embodiments, calculating of the scale can include determining aconversion algorithm/ratio (e.g., 1 foot to 100 pixels). For example, ifthe actual size of an object in realty is 2 feet tall and the determinedsize/height of the image representation of the object on the display is200 pixels, then the example embodiment 500 can determine that 1 footconverts to 100 pixels and vice versa (assuming distance is constant).In some embodiments, the determining of the scale can be based at leastin part upon the information obtained about the distance between thecomputing device and the object in reality. For example, the scale canspecify that 1 foot equals 100 pixels when the distance is 10 feet, andthat 1 foot equals 200 pixels when the distance is 15 feet. As such, insome embodiments, the scale can be variable/adjustable depending on thedistance between the computing device and the object in reality.

At step 514, the example method 500 can receive information indicativeof a physical size of a second object. At step 516, the method cangenerate an image rendering of the second object, wherein a size of theimage rendering is based on the physical size of the second object andthe scale for the captured image. For example, in the context of anonline electronic marketplace, the second object can depict and/or be arepresentation of a corresponding product in reality that is listed forsale. The image rendering of the second object can be displayed on thedisplay and can be scaled in size based at least in part upon the scale.For example, if the corresponding product in reality is 5 feet wide and3 feet tall, then the image rendering of the second object can be scaledin size and displayed on the display as being 500 pixels wide and 300pixels tall. As such, the image rendering of the second object can bepresented in a scaled size with respect to the environment and theobject on the display, which can allow the user to get a feel for whatthe actual corresponding product would look like in the environmentand/or how the actual product would fit in the environment.

FIG. 6 illustrates an example method embodiment 600 for measuringobjects in reality and viewing virtual articles. Again, it should beunderstood that there can be additional, fewer, or alternative stepsperformed in similar or alternative orders, or in parallel, within thescope of the various embodiments unless otherwise stated. The examplemethod embodiment 600 can start with capturing an image of anenvironment including an object, at step 602. The image can be capturedusing at least one camera on a computing device. The example methodembodiment 600 can display the captured image on a display of thecomputing device, at step 604.

At step 606, the example method 600 can determine a size of an imagerepresentation of the object based on the captured image. At step 608,the example method 600 can obtain information indicative of a physicalsize of the object. In some embodiments, the computing device canutilize an ultrasonic sensor, an infrared (IR) sensor, the camera,and/or other distance measuring utilities (e.g., stereoscopic cameras,RGB cameras for sensing UV light, laser beams, sonar pulses, etc.) todetermine/obtain information indicative of the physical size of theobject. In some embodiments, information indicative of the physical sizeof the object can be obtained from the user and/or from aspecified/default distance parameter (e.g., which can be stored in amemory of the device).

At step 610, the method 600 can calculate a scale for the captured imagebased at least in part on comparison between the physical size of theobject and the size of the image representation of the object. At step612, the method 600 can determine a size with which to render a virtualarticle according to the scale for the captured image, the virtualarticle being associated with a physical item. The method 600 candisplay the virtual article, rendered at the size according to thescale, on the display in conjunction with the captured image, at step614.

In some embodiments, the environment with the object(s) in reality andthe virtual article(s) can be captured and/or saved. For example, thescene of the environment, object(s), and virtual article(s) can be savedas a screen shot. The saved screen can be stored on the device and/orsent to another recipient(s) (e.g., via email, SMS text, socialnetworking, etc.). In some embodiments, the saved scene can be modifiedby moving, adding, and/or removing items/articles. In some embodiments,the environment, object(s), and virtual article(s) can be saved and/orretrieved on an online electronic marketplace.

In some embodiments, a change in rotation and/or orientation of thecomputing device can be determined by a gyroscope, an accelerometer, acompass, and/or other similar utilities on the device. The environment,the one or more objects in reality, and/or the one or more virtualarticles presented on the display screen can be adjusted accordinglybased at least in part upon the detected change in rotation and/ororientation.

Various embodiments consistent with the present disclosure can beimplemented for any camera or multiple cameras on the computing device.For example, in some embodiments, a front facing camera can be utilized.Continuing with the example, a user can hold a card of a known size infront the front facing camera. Based on the known (e.g., readilyobtainable) size of the card, the device can obtain information about(e.g., calculate, prompt for) the distance from the card to the device.Based at least in part on the known size of the card (and, in someembodiments, the distance), the device can determine a measurementscale.

FIG. 7 illustrates an example electronic user device 700 that can beused in accordance with various embodiments. Although a portablecomputing device (e.g., an electronic book reader or tablet computer) isshown, it should be understood that any electronic device capable ofreceiving, determining, and/or processing input can be used inaccordance with various embodiments discussed herein, where the devicescan include, for example, desktop computers, notebook computers,personal data assistants, smart phones, video gaming consoles,television set top boxes, and portable media players. In someembodiments, a computing device can be an analog device, such as adevice that can perform signal processing using operational amplifiers.In this example, the computing device 700 has a display screen 702 onthe front side, which under normal operation will display information toa user facing the display screen (e.g., on the same side of thecomputing device as the display screen). The computing device in thisexample includes at least one camera 704 or other imaging element forcapturing still or video image information over at least a field of viewof the at least one camera. In some embodiments, the computing devicemight only contain one imaging element, and in other embodiments thecomputing device might contain several imaging elements. Each imagecapture element may be, for example, a camera, a charge-coupled device(CCD), a motion detection sensor, or an infrared sensor, among manyother possibilities. If there are multiple image capture elements on thecomputing device, the image capture elements may be of different types.In some embodiments, at least one imaging element can include at leastone wide-angle optical element, such as a fish eye lens, that enablesthe camera to capture images over a wide range of angles, such as 180degrees or more. Further, each image capture element can comprise adigital still camera, configured to capture subsequent frames in rapidsuccession, or a video camera able to capture streaming video.

The example computing device 700 also includes at least one microphone706 or other audio capture device capable of capturing audio data, suchas words or commands spoken by a user of the device. In this example, amicrophone 706 is placed on the same side of the device as the displayscreen 702, such that the microphone will typically be better able tocapture words spoken by a user of the device. In at least someembodiments, a microphone can be a directional microphone that capturessound information from substantially directly in front of themicrophone, and picks up only a limited amount of sound from otherdirections. It should be understood that a microphone might be locatedon any appropriate surface of any region, face, or edge of the device indifferent embodiments, and that multiple microphones can be used foraudio recording and filtering purposes, etc.

The example computing device 700 also includes at least one orientationsensor 708, such as a position and/or movement-determining element. Sucha sensor can include, for example, an accelerometer or gyroscopeoperable to detect an orientation and/or change in orientation of thecomputing device, as well as small movements of the device. Anorientation sensor also can include an electronic or digital compass,which can indicate a direction (e.g., north or south) in which thedevice is determined to be pointing (e.g., with respect to a primaryaxis or other such aspect). An orientation sensor also can include orcomprise a global positioning system (GPS) or similar positioningelement operable to determine relative coordinates for a position of thecomputing device, as well as information about relatively largemovements of the device. Various embodiments can include one or moresuch elements in any appropriate combination. As should be understood,the algorithms or mechanisms used for determining relative position,orientation, and/or movement can depend at least in part upon theselection of elements available to the device.

FIG. 8 illustrates a logical arrangement of a set of general componentsof an example computing device 800 such as the device 700 described withrespect to FIG. 7. In this example, the device includes a processor 802for executing instructions that can be stored in a memory device orelement 804. As would be apparent to one of ordinary skill in the art,the device can include many types of memory, data storage, ornon-transitory computer-readable storage media, such as a first datastorage for program instructions for execution by the processor 802, aseparate storage for images or data, a removable memory for sharinginformation with other devices, etc. The device typically will includesome type of display element 806, such as a touch screen or liquidcrystal display (LCD), although devices such as portable media playersmight convey information via other means, such as through audiospeakers. As discussed, the device in many embodiments will include atleast one image capture element 808 such as a camera or infrared sensorthat is able to image projected images or other objects in the vicinityof the device. Methods for capturing images or video using a cameraelement with a computing device are well known in the art and will notbe discussed herein in detail. It should be understood that imagecapture can be performed using a single image, multiple images, periodicimaging, continuous image capturing, image streaming, etc. Further, adevice can include the ability to start and/or stop image capture, suchas when receiving a command from a user, application, or other device.The example device similarly includes at least one audio capturecomponent 812, such as a mono or stereo microphone or microphone array,operable to capture audio information from at least one primarydirection. A microphone can be a uni- or omni-directional microphone asknown for such devices.

In some embodiments, the computing device 800 of FIG. 8 can include oneor more communication elements (not shown), such as a Wi-Fi, Bluetooth,RF, wired, or wireless communication system. The device in manyembodiments can communicate with a network, such as the Internet, andmay be able to communicate with other such devices. In some embodimentsthe device can include at least one additional input device able toreceive conventional input from a user. This conventional input caninclude, for example, a push button, touch pad, touch screen, wheel,joystick, keyboard, mouse, keypad, or any other such device or elementwhereby a user can input a command to the device. In some embodiments,however, such a device might not include any buttons at all, and mightbe controlled only through a combination of visual and audio commands,such that a user can control the device without having to be in contactwith the device.

The device 800 also can include at least one orientation or motionsensor 810. As discussed, such a sensor can include an accelerometer orgyroscope operable to detect an orientation and/or change inorientation, or an electronic or digital compass, which can indicate adirection in which the device is determined to be facing. Themechanism(s) also (or alternatively) can include or comprise a globalpositioning system (GPS) or similar positioning element operable todetermine relative coordinates for a position of the computing device,as well as information about relatively large movements of the device.The device can include other elements as well, such as may enablelocation determinations through triangulation or another such approach.These mechanisms can communicate with the processor 802, whereby thedevice can perform any of a number of actions described or suggestedherein.

As an example, a computing device such as that described with respect toFIG. 7 can capture and/or track various information for a user overtime. This information can include any appropriate information, such aslocation, actions (e.g., sending a message or creating a document), userbehavior (e.g., how often a user performs a task, the amount of time auser spends on a task, the ways in which a user navigates through aninterface, etc.), user preferences (e.g., how a user likes to receiveinformation), open applications, submitted requests, received calls, andthe like. As discussed above, the information can be stored in such away that the information is linked or otherwise associated whereby auser can access the information using any appropriate dimension or groupof dimensions.

As discussed, different approaches can be implemented in variousenvironments in accordance with the described embodiments. For example,FIG. 9 illustrates an example of an environment 900 for implementingaspects in accordance with various embodiments. As will be appreciated,although a Web-based environment is used for purposes of explanation,different environments may be used, as appropriate, to implement variousembodiments. The system includes an electronic client device 902, whichcan include any appropriate device operable to send and receiverequests, messages or information over an appropriate network 904 andconvey information back to a user of the device. Examples of such clientdevices include personal computers, cell phones, handheld messagingdevices, laptop computers, set-top boxes, personal data assistants,electronic book readers and the like. The network can include anyappropriate network, including an intranet, the Internet, a cellularnetwork, a local area network or any other such network or combinationthereof. Components used for such a system can depend at least in partupon the type of network and/or environment selected. Protocols andcomponents for communicating via such a network are well known and willnot be discussed herein in detail. Communication over the network can beenabled via wired or wireless connections and combinations thereof. Inthis example, the network includes the Internet, as the environmentincludes a Web server 906 for receiving requests and serving content inresponse thereto, although for other networks an alternative deviceserving a similar purpose could be used, as would be apparent to one ofordinary skill in the art.

The illustrative environment includes at least one application server908 and a data store 910. It should be understood that there can beseveral application servers, layers or other elements, processes orcomponents, which may be chained or otherwise configured, which caninteract to perform tasks such as obtaining data from an appropriatedata store. As used herein the term “data store” refers to any device orcombination of devices capable of storing, accessing and retrievingdata, which may include any combination and number of data servers,databases, data storage devices and data storage media, in any standard,distributed or clustered environment. The application server can includeany appropriate hardware and software for integrating with the datastore as needed to execute aspects of one or more applications for theclient device and handling a majority of the data access and businesslogic for an application. The application server provides access controlservices in cooperation with the data store and is able to generatecontent such as text, graphics, audio and/or video to be transferred tothe user, which may be served to the user by the Web server in the formof HTML, XML or another appropriate structured language in this example.The handling of all requests and responses, as well as the delivery ofcontent between the client device 902 and the application server 908,can be handled by the Web server 906. It should be understood that theWeb and application servers are not required and are merely examplecomponents, as structured code discussed herein can be executed on anyappropriate device or host machine as discussed elsewhere herein.

The data store 910 can include several separate data tables, databasesor other data storage mechanisms and media for storing data relating toa particular aspect. For example, the data store illustrated includesmechanisms for storing production data 912 and user information 916,which can be used to serve content for the production side. The datastore also is shown to include a mechanism for storing log or sessiondata 914. It should be understood that there can be many other aspectsthat may need to be stored in the data store, such as page imageinformation and access rights information, which can be stored in any ofthe above listed mechanisms as appropriate or in additional mechanismsin the data store 910. The data store 910 is operable, through logicassociated therewith, to receive instructions from the applicationserver 908 and obtain, update or otherwise process data in responsethereto. In one example, a user might submit a search request for acertain type of element. In this case, the data store might access theuser information to verify the identity of the user and can access thecatalog detail information to obtain information about elements of thattype. The information can then be returned to the user, such as in aresults listing on a Web page that the user is able to view via abrowser on the user device 902. Information for a particular element ofinterest can be viewed in a dedicated page or window of the browser.

Each server typically will include an operating system that providesexecutable program instructions for the general administration andoperation of that server and typically will include computer-readablemedium storing instructions that, when executed by a processor of theserver, allow the server to perform its intended functions. Suitableimplementations for the operating system and general functionality ofthe servers are known or commercially available and are readilyimplemented by persons having ordinary skill in the art, particularly inlight of the disclosure herein.

The environment in one embodiment is a distributed computing environmentutilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 9. Thus, the depiction of the system 900 in FIG. 9should be taken as being illustrative in nature and not limiting to thescope of the disclosure.

As discussed above, the various embodiments can be implemented in a widevariety of operating environments, which in some cases can include oneor more user computers, computing devices, or processing devices whichcan be used to operate any of a number of applications. User or clientdevices can include any of a number of general purpose personalcomputers, such as desktop or laptop computers running a standardoperating system, as well as cellular, wireless, and handheld devicesrunning mobile software and capable of supporting a number of networkingand messaging protocols. Such a system also can include a number ofworkstations running any of a variety of commercially-availableoperating systems and other known applications for purposes such asdevelopment and database management. These devices also can includeother electronic devices, such as dummy terminals, thin-clients, gamingsystems, and other devices capable of communicating via a network.

Various aspects also can be implemented as part of at least one serviceor Web service, such as may be part of a service-oriented architecture.Services such as Web services can communicate using any appropriate typeof messaging, such as by using messages in extensible markup language(XML) format and exchanged using an appropriate protocol such as SOAP(derived from the “Simple Object Access Protocol”). Processes providedor executed by such services can be written in any appropriate language,such as the Web Services Description Language (WSDL). Using a languagesuch as WSDL allows for functionality such as the automated generationof client-side code in various SOAP frameworks.

Most embodiments utilize at least one network that would be familiar tothose skilled in the art for supporting communications using any of avariety of commercially-available protocols, such as TCP/IP, OSI, FTP,UPnP, NFS, CIFS, and AppleTalk. The network can be, for example, a localarea network, a wide-area network, a virtual private network, theInternet, an intranet, an extranet, a public switched telephone network,an infrared network, a wireless network, and any combination thereof.

In embodiments utilizing a Web server, the Web server can run any of avariety of server or mid-tier applications, including HTTP servers, FTPservers, CGI servers, data servers, Java servers, and businessapplication servers. The server(s) also may be capable of executingprograms or scripts in response requests from user devices, such as byexecuting one or more Web applications that may be implemented as one ormore scripts or programs written in any programming language, such asJava®, C, C# or C++, or any scripting language, such as Perl, Python, orTCL, as well as combinations thereof. The server(s) may also includedatabase servers, including without limitation those commerciallyavailable from Oracle®, Microsoft®, Sybase®, and IBM®.

The environment can include a variety of data stores and other memoryand storage media as discussed above. These can reside in a variety oflocations, such as on a storage medium local to (and/or resident in) oneor more of the computers or remote from any or all of the computersacross the network. In a particular set of embodiments, the informationmay reside in a storage-area network (“SAN”) familiar to those skilledin the art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate. Where a system includescomputerized devices, each such device can include hardware elementsthat may be electrically coupled via a bus, the elements including, forexample, at least one central processing unit (CPU), at least one inputdevice (e.g., a mouse, keyboard, controller, touch screen, or keypad),and at least one output device (e.g., a display device, printer, orspeaker). Such a system may also include one or more storage devices,such as disk drives, optical storage devices, and solid-state storagedevices such as random access memory (“RAM”) or read-only memory(“ROM”), as well as removable media devices, memory cards, flash cards,etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed, and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules, services, or other elementslocated within at least one working memory device, including anoperating system and application programs, such as a client applicationor Web browser. It should be appreciated that alternate embodiments mayhave numerous variations from that described above. For example,customized hardware might also be used and/or particular elements mightbe implemented in hardware, software (including portable software, suchas applets), or both. Further, connection to other computing devicessuch as network input/output devices may be employed.

Storage media and computer readable media for containing code, orportions of code, can include any appropriate media known or used in theart, including storage media and communication media, such as but notlimited to volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules, or other data, including RAM, ROM, EEPROM, flash memoryor other memory technology, CD-ROM, digital versatile disk (DVD) orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bythe a system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

What is claimed is:
 1. A computer-implemented method comprising:receiving a first image, the first image including a representation of aphysical environment and a representation of a first physical objectpresent in the physical environment; determining a size of therepresentation of the first physical object included in the first image;determining, based at least in part on data from a laser sensor, adistance between a display and the first physical object present in thephysical environment; calculating a physical size of the first physicalobject based on the distance and the size of the representation of thefirst physical object; calculating a conversion scale based on thephysical size of the first physical object and the size of therepresentation of the first physical object; receiving informationrelating to a physical size of a second object; determining, based onthe physical size of the second object, that the second object fits aspace in the physical environment; generating a second image includingthe representation of the physical environment and a representation ofthe second object rendered at a size based on the physical size of thesecond object and the conversion scale; and displaying the second imageon the display.
 2. The computer-implemented method of claim 1, furthercomprising: receiving second information relating to at least one of acolor, a pattern or a style; generating a third image including therepresentation of the physical environment and a second representationof the second object rendered based on at least one of the color, thepattern, or the style; and displaying the third image on the display. 3.The computer-implemented method of claim 1, further comprisingdetermining a dimension of the representation of the first physicalobject in pixels.
 4. The computer-implemented method of claim 1, furthercomprising analyzing at least one of ultrasonic sensor data, infraredsensor data, a parameter stored in a memory, stereoscopic image datafrom a stereoscopic camera, sonar pulse data, RGB image data from an RGBcamera, or a user input.
 5. The computer-implemented method of claim 1,further comprising determining a ratio between the physical size of thefirst physical object and the size of the representation of the firstphysical object.
 6. The computer-implemented method of claim 1, whereinthe second image further includes the representation of the firstphysical object.
 7. A computer-implemented method comprising: receivinga first image, the first image including a representation of a physicalenvironment and a representation of a physical object present in thephysical environment; determining a first size of the representation ofthe physical object included in the first image; determining, based atleast in part on data from a laser sensor, a distance between a displayand the physical object present in the physical environment; calculatinga physical size of the physical object based at least in part on thedistance and the first size; calculating a conversion scale based atleast in part on the physical size of the physical object and the firstsize of the representation of the physical object; receiving secondinformation relating to a physical size of a physical item; determining,based on the physical size of the physical item, that the physical itemfits a space in the physical environment; determining a second size of avirtual article representing the physical item based at least in part onthe conversion scale and the physical size of the physical item; andgenerating a second image including the representation of the physicalenvironment and the virtual article rendered based on the second size.8. The computer-implemented method of claim 7, further comprising:receiving at least one of a color, a pattern, or a style; and generatinga third image including the representation of the physical environmentand the virtual article rendered based on at least one of the color, thepattern, or the style.
 9. The computer-implemented method of claim 7,further comprising receiving a user input relating to the physical sizeof the physical object.
 10. The computer-implemented method of claim 7,wherein the physical item includes at least one of a piece of furniture,an appliance, a fixture, a wall, a piece of art, a decorative item, or atextile.
 11. The computer-implemented method of claim 7, furthercomprising: sending the second image to one or more users.
 12. Thecomputer-implemented method of claim 7, further comprising: receivingthird information relating to a change in a distance between a displayand the physical object; and adjusting the conversion scale based atleast in part on the change in the distance between the display and thephysical object.
 13. The computer-implemented method of claim 7, furthercomprising: receiving third information relating to a change of aposition of the virtual article in the representation of the physicalenvironment; and generating a third image including the representationof the physical environment and the virtual article rendered based onthe change of the position.
 14. The computer-implemented method of claim13, wherein the virtual article is further rendered based on at leastone of change of a rotation, a change of a translation, or a change of ascale of the virtual article.
 15. A computing device comprising: atleast one camera; a display; a processor; and a memory device includinginstructions that, upon being executed by the processor, cause thecomputing device to: receive a first image, the first image including arepresentation of a physical environment and a representation of aphysical object present in the physical environment; determine a firstsize of the representation of the physical object included in the firstimage; determine, based at least in part on data from a laser sensor, adistance between the display and the physical object present in thephysical environment; calculate a physical size of the physical objectbased at least in part on the distance and the first size; calculate aconversion scale based at least in part on the physical size of thephysical object and the first size of the representation of the physicalobject; receive second information relating to a physical size of aphysical item; determine, based on the physical size of the physicalitem, that the physical item fits a space in the physical environment;determine a second size of a virtual article based at least in part onthe conversion scale and the physical size of the physical item; andgenerate a second image including the representation of the physicalenvironment and the virtual article rendered based on the second size.16. The computing device of claim 15, wherein the instructions uponbeing executed further cause the computing device to: receive at leastone of a color, a pattern, or a style; and generate a third imageincluding the representation of the physical environment and the virtualarticle rendered based on at least one of the color, the pattern, or thestyle.
 17. The computing device of claim 15, further comprising: atleast one of an ultrasonic sensor, an infrared sensor, an ultravioletsensor, a stereoscopic camera, a RGB camera, or a sonar pulse sensorcapable of being utilized for determining the distance between thedisplay and the physical object.
 18. The computing device of claim 15,wherein the instructions upon being executed further cause the computingdevice to: determine the distance between the display and the physicalobject based at least in part on at least one of a parameter stored inthe memory device or a user input.
 19. The computing device of claim 15,further comprising: a transceiver configured to send the second image toone or more users.